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WE CLAIM: 



/ 1 . Method for generating facial animation values using a sequence of facial image 
frames and synchronously \aptured audio data of a speaking actor, comprising the steps for: 

providing a plurality \f visual-facial-animation values based on tracking of facial features 
in the sequence of facial imaga frames of the speaking actor; 

providing a plurality of iudio-facial-animation values based on visemes detected using 
the synchronously captured audid voice data of the speaking actor; and 

combining the plurality ofVisual facial animation values and the plurality of audio facial 
animation values to generate output facial animation values for use in facial animation. 

2. Method for generating facial animation values as defined in claim 1, wherein the 
output facial animation values associated with a mouth for a facial animation are based only on 
the respective mouth-associated values of the plurality of audio facial animation values. 

3. Method for generating facial anmiation values as defined in claim 1, wherein the 
output facial animation values associated wrth a mouth for a facial animation are based on a 
weighted average of the respective mouth-associated values of the plurality of visual facial 
animation values and the respective mouth-associated values of the plurality of audio facial 
animation values. \ 

4. Method for generating facial animation Values as defined in claim 1, wherein the 
output facial animation values associated with a mouth for a facial animation are based on 
Kalman filtering of the respective mouth-associated values of the plurality of visual facial 
animation values and the respective mouth-associated Values of the plurality of audio facial 
animation values. \ 

5. Method for generating facial animation values $s defined in claim 1, wherein the step 
of combining the plurality of visual facial animation valuesWd the plurality of audio facial 
animation values to generate output facial animation values includes detecting whether speech is 




Gray Cary\GT\6251897.1 
103353-990000 



Docket No. EYEM1340 



# 



7 



occurring in the synchronously captured audio voice data of the speaking actor and, while speech 
is detected as occurring, generating the output facial animation values associated with a mouth 
based only on the respective mouth-associated values of the plurality of audio facial animation 
values and, while speech is not detected as occurring, generating the output facial animation 
values associated with a mouth based only on the respective mouth- associated values of the 
plurality of visual facial animation values. 

6. Method for generatingYacial animation values as defined in claim 1, wherein the 
tracking of facial features in the sequence of facial image frames of the speaking actor is 
performed using bunch graph matchmg. 

7. Method for generating facial animation values as defined in claim 1, wherein the 
tracking of facial features in the sequenceVf facial image frames of the speaking actor is 
performed using transformed facial image flames generated based on wavelet transformations. 

8. Method for generating facial animation values as defined in claim 1, wherein the 
tracking of facial features in the sequence of facial image frames of the speaking actor is 
performed using transformed facial image frames generated based on Gabor wavelet 
transformations. \ 

9. Apparatus for generating facial animation values using a sequence of facial image 
frames and synchronously captured audio data of a speaking actor, comprising: 

means for providing a plurality of visual-facial-anrmation values based on tracking of 
facial features in the sequence of facial image frames of the\speaking actor; 

means for providing a plurality of audio-facial-animafaon values based on visemes 
detected using the synchronously captured audio voice data ofthe speaking actor; and 

means for combining the plurality of visual facial animation values and the plurality of 
audio facial animation values to generate output facial animation Values for use in facial 
animation. \ 
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10. Apparatus for generating facial animation values as defined in claim 9, wherein the 
output facial animation values associated with a mouth for a facial animation are based only on 
the respective moutMassociated values of the plurality of audio facial animation values. 

1 1 . Apparatus for generating facial animation values as defined in claim 9, wherein the 
output facial animation values associated with a mouth for a facial animation are based on a 
weighted average of the respective mouth-associated values of the plurality of visual facial 
animation values and the respective mouth-associated values of the plurality of audio facial 
animation values. \ 

12. Apparatus for generating facial animation values as defined in claim 9, wherein the 
output facial animation values associated with a mouth for a facial animation are based on 
Kalman filtering of the respective mWh-associated values of the plurality of visual facial 
animation values and the respective mouth- associated values of the plurality of audio facial 
animation values. \ 

13. Apparatus for generating facialVnimation values as defined in claim 9, wherein the 
means for combining the plurality of visual racial animation values and the plurality of audio 
facial animation values to generate output facial animation values includes means for detecting 
whether speech is occurring in the synchronously captured audio voice data of the speaking actor 
and, while speech is detected as occurring, generating the output facial animation values 
associated with a mouth based only on the respective mouth-associated values of the plurality of 
audio facial animation values and, while speech is nokdetected as occurring, generating the 
output facial animation values associated with a moutnbased only on the respective mouth- 
associated values of the plurality of visual facial animation values. 

14. Apparatus for generating facial animation values\as defined in claim 9, wherein the 
tracking of facial features in the sequence of facial image frames of the speaking actor is 
performed using bunch graph matching. \ 
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ng facial animation values as defined in claim 9, wherein the 
tracking of facial features in the sequence of facial image frames of the speaking actor is 
performed using transformed facia? image frames generated based on wavelet transformations. 



16. Apparatus for generating facial animation values as defined in claim 9, wherein the 
tracking of facial features in the. sequence of facial image frames of the speaking actor is 
performed using transformed facial irn^ge frames generated based on Gabor wavelet 
transformations. 
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